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Although afterschool programs for children have been operating for many years in some 
communities, the aherschool movement - the great national awakening to the opportunity 
afterschool offers - is just a few years old. As public demand for afterschool has grown, so has 
the demand for accountability. That is particularly true in afterschool programs that spend public 
dollars. After all, where tax dollars flow, so must accountability to taxpayers. 

Fortunately for afterschool advocates, a number of afterschool evaluations are showing gains for 
children, especially those who regularly participate in afterschool programs and those at highest 
risk of academic failure. This updated evaluations backgrounder includes new data from 
ongoing evaluations of LA’s BEST and Citizen Schools, as well as recently released evaluations 
of the YMCA of Greater New York’s Virtual Y Program, the Young Scholars Program, 
Generacion Diez, 2L' Century Community Learning Centers throughout Texas and findings 
from a Yale University study and the Massachusetts After-School Research Study. Finally, 
highlights from previous evaluations of well known afterschool programs such as TASC, 
Foundations Inc., Project Learn, San Diego’s ‘6 to 6’ and more are included. A list of the studies 
and their key findings is provided at the end of this document on Page 27. 

This compilation focuses chiefly on the impact of afterschool programs on student academic 
achievement. A second backgrounder, available from the Afterschool Alliance website at 
www.afterschoolalliance.org, summarizes findings related to student safety, behavior, substance- 
abuse-prevention, and discipline. 



The Landscape of Afterschool Evaluations 

A number of different types of evaluations have been conducted over the last several years, 
assessing various aspects of afterschool programming. Some evaluations seek to gather data on 
whether programs have been structured as they were originally intended, how well they have 
done at meeting ahendance and staffing goals, how they “fit” in the school environment and 
more. Others explore student and other outcomes - the effects afterschool programs have on the 
children who participate in them, their parents, and even the communities at large. 

Both types of evaluations are of great value to afterschool providers and to policymakers, and 
when taken together the two types of studies help identify the particular program elements and 
approaches most critical to accomplishing program goals. For example, studies correlating 
increased afterschool ahendance with increased academic performance have triggered 



www.afterschoolaUiance.org 



1 



considerable discussion in the afterschool community about ways to improve student attendance 
in aftersehool programs. 

Evaluations also differ by virtue of who conducts them. Many programs self-evaluate, providing 
useful data and satisfying the needs of their various stakeholders - parents, funders, partnering 
businesses, local public officials and so on. But for academics and large funders - the federal 
government, state governments, the Open Society Institute, the Charles Stewart Mott Foundation 
or the Wallace Fund, for example - more exacting standards and greater independence is often 
required. Independent evaluations commissioned by sueh entities are the primary subjeet of this 
doeument. 



Summary Lessons from the Data 

In reviewing the numerous studies ineluded in this backgrounder, a few key themes emerged. 
Below we present the summary lessons from the studies included in this backgrounder. The data 
and conclusions from the studies amply demonstrate the positive effect afterschool programs 
have on student aeademie achievement. Detailed deseriptions and findings from all the studies, 
including citations, are ineluded in the Afterschool Evaluations in Detail section (Page 5). See 
Appendix A, Afterschool Evaluations at a Glance, on Page 27 for a summary listing of the 
studies included in this backgrounder. 



Improved School Attendance and Engagement in Learning 

> Evaluations of FA’s BEST by the UCFA Center for the Study of Evaluation revealed that 
students’ regular school-day attendance improved once they began participating in the 
afterschool program. Students also reported higher aspirations regarding finishing school 
and going to college. In a longitudinal study, researchers at UCLA found drop out rates 
among LA’s BEST students are 20% lower than the overall distriet drop out rate. 

> Research by the Texas State Education Agency found that strong participation in 2U‘ CCLC 
afterschool programs correlated with better attendanee during the regular school day. 
“Approximately half (48%) of youth who participated in three quarters or more of the 
available 21st CCLC activities missed 5 or fewer days of school during the fall semester, 
compared to 17% of youth who participated in less than one quarter of the available 
activities. 

> Policy Studies Associates five year evaluation of The After-School Corporation (TASC) 
afterschool programs found that regular school day attendance for the pre-K to eighth grade 
levels increased for participants compared to non-participants, especially for students in 
grades 5-8, and attendance for seventh and eighth grade participants increased by 2.7 school 
days in comparison to non-participants. At the high school level, regular sehool day 
attendanee for students in the lowest quartile of attendance inereased by 4.4 days, compared 
to non-participants who were also in the lowest attendance quartile. 

> Pathways to Progress students in St. Paul, Minnesota experienced dramatically better school 
attendance — participants attended 18.44 more school days and missed 9.57 fewer school 
days than their nonparticipant peers. 
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> An evaluation of Boys & Girls Clubs’ Project Learn found that engagement in reading, use of 
verbal skills, writing, tutoring, and the study of geography all significantly increased as 
program involvement increased. 

> Students participating in California’s After School Education and Safety Program (formerly 
the After School Learning and Safe Neighborhoods Partnerships Program, ASLSNPP) 
improved their regular day attendance; with particularly large improvements for students 
with the lowest attendance record prior to the start of the program. Students also 
demonstrated a more positive attitude toward school, enhanced confidence about learning 
and increased educational aspirations. 

> Absences among Young Scholars participants decreased by 48% when they began 
participating in the afterschool program. 

Improved Test Scores and Grades 

> Students participating in LA’s BEST afterschool programs demonstrated higher academic 
achievement on standardized tests of math, reading and language arts. In addition, language 
redesignation rates favored LA’s BEST students when compared with non-LA’s BEST 
students. 

> Prior to participating in Young Scholars, many of the participants were retained or required 
to repeat a grade each year. Over five years, promotion rates for Young Scholars improved 
by 83%. Young Scholars participants also made significant gains on North Carolina’s state 
tests. Young Scholars with at least 280 hours in the program averaged double-digit increases 
annually for proficiency in both math and reading. Eurthermore, the number of Young 
Scholars receiving A’s and B’s increased an average of 38%, while the number receiving E’s 
decreased an average of 50%. 

> Children in the Ohio Urban School Initiative School Age Child Care Project (SACC) scored 
higher in every subject area tested (writing, math, reading, citizenship and science) than non- 
participating students from across the state, according to an evaluation by the University of 
Cincinnati College of Education’s Evaluation Services Center. 

> An evaluation by the Department of Education, University of California at Irvine and 
Research Support Services, found that reading and mathematics gains of students in Los 
Angeles’ YS-CARE program, aimed at children from families on LANE, outpaced those of 
non-participating students, as measured by SAT-9 scores. 

> Policy Studies Associates’ second-year evaluation of The After-School Corporation’s 
(TASC’s) program found significant differences in proficiency-level shifts among active 
participants and nonparticipants who scored in the lowest proficiency level on the 1998-99 
mathematics tests. When tested in 1999-2000, participants were more likely than non- 
participants to score at a higher proficiency level as compared to their 1998-99 performance. 
A similar but less pronounced pattern was observed on the reading tests administered in 
grades 3-8. 

> Reading scores for San Diego’s “6 to 6” students improved, according to evaluator WestEd. 
Fifty-seven percent of students increased their reading scores over the course of the studied 
year, and SAT-9 reading scores increased. Nearly ten percent of children moved up into the 
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25^*' percentile or higher in 2000 by comparison to 1999 reading scores. In addition, 44% of 
students in San Diego’s “6 to 6” program increased their SAT-9 math scores. 

> A five-site evaluation of the Boys & Girls Clubs’ national Project Learn program found 
average grade increases over the 30-month study period were greatest for program youth - 
afterschool participants increased their average grades by 11% while comparison youth 
increased their average grades by only .4%. 

> Students participating in Foundations, Inc afterschool programs scored higher on math, 
reading, and language arts standardized tests, at every grade level, than the comparison 
group. 

Frequency and Duration of Afterschool Participation Increases Benefits 

> LA’s BEST students who participated most frequently and for the longest period of time 
were least likely to drop out of school. 

> Policy Studies Associates’ found that students who participated in TASC after-school 
activities the most consistently and for the longest period of time experienced the greatest 
math gains. 

> A statewide evaluation of California’s After School Education and Safety Program (formerly 
the After School Learning and Safe Neighborhoods Partnerships Program, ASLSNPP) by the 
University of California at Irvine demonstrated mathematics gains closely related to 
individual students’ levels of participation in the program. 

> The Texas State Education Agency found that academic progress was closely linked to the 
extent of participation in afterschool — fewer than half (46%) of youth who attended 25% or 
less of available afterschool reading tutorials showing improved reading ability at the end of 
the semester, compared to 64% of youth who attended more than 75% of reading tutorials. 

> A Yale University study of afterschool participation found that children who were in the 
highest category of afterschool program attendance had significantly higher reading 
achievement than children in all other care arrangements (parent, sibling/self-care or some 
combination) while children at lower levels of participation outperformed children in only 
some of the other care arrangements. 

> An external evaluation of the Afterschool Education and Safety Program in Santa Ana, CA 
found that students who attended the afterschool program more frequently (38 or more days 
per year) showed better results than students who attended less frequently or not at all. 



Students at Greatest Risk Show Greatest Gains 

> The TASC program evaluation concluded that participants who were at greatest academic 
risk made the largest math gains, when compared to other students. Math benefits were most 
clearly evident for students who scored in the lowest of four proficiency levels in the year 
prior to TASC participation. Eurthermore, among students from low-income families, the 
evaluation also found evidence of after-school benefits in math after two or more years of 
active participation. 
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> North Carolina’s Support Our Students ’ participants who were the furthest behind and had 
the most risk faetors (e.g., free/redueed luneh status, single-parent households, ete.) made the 
greatest gains on their End of Grade (EOG) Achievement Test. 

> An external evaluation of Adams County, Pennsylvania’s Generaeion Diez afterschool 
program found significantly greater gains in a number of areas (spelling achievement, math 
achievement, reading achievement) for participating children whose families were less 
engaged in sehool and/or were functioning at a lower level. 



Afterschool Evaluations in Detail 

Over the past decade a number of important afterschool evaluations have been conducted - more 
than enough to demonstrate that afterschool programs help children achieve, hollowing are 
summaries of several of the most extensive evaluations. The summaries are organized by scope, 
beginning with a meta-analysis covering evaluations of afterschool programs and evaluations of 
programs that are national in scope. We then turn to evaluations of programs operating at a state 
level and end with local or program level evaluations. Studies that are newly included or 
updated for this 2006 version of the evaluations backgrounder are indicated as such. 

National Studies 

The Mid-continent Research for Education and Learning (McREL) Meta-Analysis 

In January 2004, Mid-continent Research for Education and Learning (McREL), released a meta- 
analysis of 53 separate studies of out-of-school time programs, after sifting through hundreds of 
studies spanning many years. The study is available at 

http://www.merel.org/PDF/SchoolImprovementReform/5032RR RSOSTeffeetiveness.pdf , and 
is cited here as McREL Study. A meta-analysis is a statistical re-analysis of the results of several 
studies on a related topic, conducted for the purpose of integrating the studies’ findings, 
notwithstanding differences in their respective research methodologies. The studies selected for 
inclusion in the McREL meta-analysis all met rigorous methodological standards. The study was 
conducted for the Institute of Education Sciences of the U.S. Department of Education on a 
federal contract. 

McREL ’s findings affirmed the widely held view that afterschool programs help students 
aehieve in school. McREL’s researchers summarized their findings: 

The synthesis resulted in statistically significant positive effects of OST [out of 
school time] on both reading and mathematics student achievement. The overall 
effect sizes ranged from .06 to .13 for reading and from .09 to .17 for mathematies, 
depending on the statistical model used for meta-analysis. Though numerieally 
small, these results are important because they are based on strategies to supplement 
the regular school day and to prevent learning loss. Positive findings for 
supplementary programs that address the needs of low-achieving or at-risk students 
are therefore encouraging. Together, the results for reading and mathematies 
suggest that OST programs can significantly increase the achievement of these 
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students by an average of one-tenth of a standard deviation compared to those 
students who do not participate in OST programs. . . . [McREL Study, page 2] 

In addition to the analyses of study outcomes, the syntheses of reading and 
mathematics studies described some common features among the studies in each 
content area. In reading, these were the links between student attendance and 
student achievement, the importance of staff quality, the development of academic 
and social skills, the implementation of a well-defined reading curriculum, and the 
prevention of learning loss. Common features highlighted in the mathematics 
studies were additional time for remediation, the use of tutoring, the use of 
counseling and mentoring, and the combination of recreation with mathematics 
instruction. Overall, the meta-analytic and narrative results lead to the following 
conclusions and implications for practice and policy related to OST and its 
evaluation; 

> OST strategies can have positive effects on the achievement of low-achieving or at- 
risk students in reading and mathematics. 

> The timeframes for delivering OST programs (i.e., after school or summer school) do 
not influence the effectiveness of OST strategies. 

> Students in early elementary grades are more likely than older elementary and middle 
school students to benefit from OST strategies for improving reading, while there are 
indications that the opposite is true for mathematics. 

> OST strategies need not focus solely on academic activities to have positive effects 
on student achievement. 

> Administrators of OST programs should monitor program implementation and 
student learning in order to determine the appropriate investment of time for specific 
OST strategies and activities. 

> OST strategies that provide one-on-one tutoring for low-achieving or at-risk students 
have strong positive effects on student achievement in reading. [McREL Study, 
pages 2-3] 



The Boys & Girls Clubs’ Project Learn 

Begun in 1996, the Boys & Girls Clubs’ Project Learn focuses on providing youngsters with 
“high-yield learning activities,” including weekly discussions with knowledgeable adults, leisure 
reading, writing activities, homework help, helping others, and games that rely on cognitive 
skills. The program has been implemented in full at one-tenth of the Clubs’ 3,300 sites, and all 
sites are implementing components of the program. Steven P. Schinke, Ph.D., of New York’s 
Columbia School of Social Work led an evaluation of the program that relied on a quasi- 
experimental design. Three groups of students were identified, in five separate cities. The 
groups: 
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• Students participating in Boys & Girls Clubs programs that had implemented Project 
Learn, (BGC program sites), 

• In the same cities, students at Boys & Girls Clubs that had not implemented Project Learn 
(BGC comparison sites), and 

• In the same cities, students at non-Boys & Girls Club sites that had not implemented the 
kind of enhanced learning initiatives characteristic of Project Learn (non-BGC 
comparison sites). 

All students in all groups lived in public housing projects, and the sites were chosen to be 

nationally representative of students in public housing. Data on students’ academic performance 

were collected four times: before they began the program, six months after they began, 18 

months after they began, and 30 months after they began. Findings included: 

> “The level of program involvement, as rated by teachers on a scale of 0 to 10, was found to 
be associated with a number of self-reported academic outcomes.” [A Profile of the 
Evaluation of the Boys & Girls Clubs of America — Project Learn/Educational Enhancement 
Program, Harvard Family Research Project, 

http://www.gse.harvard.edu/hfrp/proiects/afterschool/mott/pleep.html . Based on Enhancing 
the Educational Achievement of At-Risk Youth, Steven P. Schinke, Ph.D. et. al. 

> “As program involvement increased, engagement in reading, use of verbal skills, writing, 
tutoring, and the study of geography all significantly (p<.05) increased as well.” 

> “There was also a direct and statistically significant (p<.05) relationship between program 
involvement and enjoyment of reading, use of verbal skills, writing, and geography.” 

> “At final follow-up (30 months after the program began), program youth more than BGC 
comparison youth and comparison youth more than non-BGC youth reported greater 
engagement in reading, enjoyment of reading, engagement in verbal activities, enjoyment of 
verbal activities, engagement in writing, enjoyment of writing, engagement in tutoring, 
enjoyment of tutoring, and enjoyment of geography.” 

> “Also at 30-month data collection, relative to the non-BGC comparison group youth, 
program and BGC comparison youth reported greater study of geography, more engagement 
in board games, enjoyment of board games, engagement in life-enhancement activities, and 
enjoyment of life-enhancement activities.” 

> “Data from teacher reports at final follow-up reveal that program and BGC comparison youth 
more than non-BGC comparison youth had more positive reading skills, writing skills, games 
skills, overall school performance, and interest in class material.” 

> “School grades at 30-month follow-up favored program youth over BGC comparison youth 
and non-BGC comparison youth on overall averages, reading scores, spelling scores, history 
scores, science scores, social studies scores, and attendance. At the 30-month follow-up, 
program and BGC comparison youth had better grades in math than non-BGC comparison 
youth.” 

> “Average grade increases over the 30-month study period were greatest for program youth. 
Program youth increased their average grades by 1 1 percent from baseline to the 30-month 
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measurement while BGC eomparison youth and non-BGC eomparison youth, over the same 
period, inereased their average grades by .4 pereent and .3 pereent, respeetively.” 

> “Program youth missed an average of only 2.19 days of sehool a year at the 30-month 
measurement as eompared to missing an average of 6.4 days a year at baseline. In eontrast, 
BGC comparison youth went from missing an average of 4.85 days of school in the baseline 
year to missing an average of 12.33 days a year at the 30-month follow-up. Similarly, non- 
BGC youth went from 7.47 days at baseline to 16.67 at follow-up. The differences between 
the school attendance of program youth and youth in both comparison groups at 30 months 
were statistically significant at p<.05, while they had not been significantly different at 
baseline.” [All bullets from HFRP Profile.] 

State-Level Evaluations 



Young Scholars Program, North Carolina 

In 1999, the Z Smith Reynolds Foundation (ZSRF) came to the conclusion that extended day 
programs could go far beyond their original goals and not only provide supervision during 
afterschool hours but also provide an academic boost. Working with the Public School Forum, 
ZSRF envisioned a program that would combine the best elements of traditional extended day 
programs with academic support that would engage young people’s minds. The result was the 
Young Scholars Program. From 2000-2005, Young Scholars Programs across the state of North 
Carolina served nearly 1,000 young people in 19 elementary and middle schools. 

In August 2006, ZSRF released a report documenting the program’s outcomes. That report, 
available at http://www.ncforum.org/doclib/forum report/collateralAfSP Aug2006.pdf, found 
that: 

> Young Scholars participants made significant gains on the state’s ABC tests. Young 
Scholars with at least 280 hours in the program averaged double-digit increases annually for 
proficiency in both math and reading. Throughout the five-year period, participants in 
Young Scholar programs consistently exceeded the state’s expected rate of academic growth. 

> Prior to participating in Young Scholars, many of the participants were retained or required 
to repeat a grade each year. Over the five year time covered by this study, promotion rates 
for Young Scholars improved by 83%. 

> Young Scholars school attendance improved substantially - absences among participants 
decreased by 48%. 

> Young Scholars participation during the regular day increased dramatically, as did their 
grades. The number of Young Scholars receiving A’s and B’s increased an average of 38% 
while the number receiving F’s decreased an average of 50%. 

> Young Scholars parents became far more involved in school activities. The longer students 
were involved in the program, the more involved parents became. 
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21st Century Community Learning Centers - Texas 

At the end of the 2003-2004 school year, Texas had 32 21^^^ Century Community Learning Center 
grantees, operating 136 afterschool sites. Relying on pre-program and post-program data 
collected from sites as part of grant reporting requirements, the Texas Education Agency, the 
state’s department of education, compiled and analyzed data on student performance and 
program implementation. A summary of the evaluation’s findings by the Harvard Family 
Research Project is available at 

http://www.gse.harvard.edu/~hfrp/proiects/afterschool/mott/21stcclctx.pdf Among the TEA’s 
chief findings: 

Among participating students, academic progress was closely linked to the extent of participation 
in afterschool. “Eess than half (46%) of youth who attended 25% or less of available tutorials 
showed improved reading ability at the end of the semester, compared to 64% of youth who 
attended more than 75% of reading tutorials. Conversely, 40% of participants who attended less 
than one quarter of the fall reading tutorials experienced a decline in their reading ability, as 
measured by pretests and posttests, compared to just 20% of youth who attended more than 75% 
of the reading tutorials. Similar results were observed for reading in the spring 2004 term. After 
controlling for demographic factors, youth who participated in 26%-50%, 51%-75%, and over 
75% of available spring reading tutorials showed more increased reading ability than youth 
attending less than 25%. These relationships were all statistically significant (p < .05).” [“A 
Profile of the Evaluation of 21st Century Community Learning Centers — Texas,” page 9, 

Harvard Family Research Project, 

http://www.gse.harvard.edu/~hfrp/proiects/afterschool/mott/21stcclctx.pdf 1 Similar findings 
were reported for mathematics and science, especially during the fall semester. 

Strong participation in afterschool correlated with better attendance during the regular school 
day. “Approximately half (48%) of youth who participated in three quarters or more of the 
available 2L' CCEC activities missed five or fewer days of school during the fall semester, 
compared to 17% of youth who participated in less than one quarter of the available activities. 
The corresponding percentages for spring were 33% and 26%, respectively. Youth who 
participated in 50% or more of the available 2L* CCEC activities were absent approximately two 
regular school days less in the spring term than youth who participated in less than 50% of the 
available 2L* CCEC activities. This difference was statistically significant (p < .01). After 
controlling for demographic factors, those youth participating in 26%-50%, 51%-75%, and 
75%-100% of available activities missed significantly fewer days of school than those 
participating in 25% or fewer activities (p < .01).” [HERE Profile, page 10.] 




The Massachusetts After-School Research Study (MARS) 



In 2005, the Intercultural Center for Research in Education and the National Institute on Out-of- 
School Time at Wellesley College published the results of the Massachusetts After-School 
Research Study (MARS), which examined the track records of 78 different Massachusetts 
afterschool centers serving more than 4,100 children. The study’s objective was to “identify 
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those program characteristics that are most closely related to high quality implementation, and to 
explore the links between program quality and youth outcomes.” [MARS, p. 1, 
http://www.wcwonline.Org/mars/MARSfull.pdf .1 

Among the study’s findings: 

> “Staff make a difference in program quality, and staff who have a strong educational 
background and appropriate training are key to program quality. At the same time, given 
current wages and working conditions, many programs can not attract such staff, or even if 
recruited, lose a significant portion of staff each year.” 

> “A highly qualified coordinator or program director is key to setting the tone for a program 
that promotes youth engagement, staff engagement, and the quality of activities and 
homework.” 

> “Most programs in the MARS sample had very low staff-to-child ratios, typically between 
1:7 and 1:9. We found clear links between low ratios and high quality, as has previous 
research in the field. At the same time, where ratios are very low, programs may want to 
consider the trade-off between paying higher salaries and having additional staff.” 

> “One of the key youth outcomes — relations with adults — ^was positively associated with the 
quality of family relations we observed at pick-up time. Programs rated high on this 
component of the APT also tended to cite parent, community and volunteer support as 
strengths of their programs. This finding suggests that all afterschool providers could benefit 
from paying attention to their relationships with the families of the youth in their programs.” 



After School Education and Safety Program - California 

Begun in 1998 as the California Afterschool Learning and Safe Neighborhoods Partnerships 
Program, the After School Education and Safety Program (ASESP) provides $117 million 
annually in matching funds to local partnerships of school districts, community groups, and local 
governments to provide before and afterschool programs for students. In selecting grantees, the 
state gives priority to schools where 50% or more of pupils are eligible for free or reduced-cost 
meals. Programs are designed locally, but are required to include an “educational and literacy 
component to provide tutoring or homework assistance in one or more of the following subject 
areas: language arts, mathematics, history and social science, or science”; and “an educational 
enrichment component, which may include but is not limited to, recreation and prevention 
activities. Such activities might involve the arts, music, physical activity, health promotion, and 
general recreation; work preparation activities; community service-learning; and other youth 
development activities based on student needs and interests.” [California ’s Before and After 
School Learning and Safe Neighborhoods Partnerships Program Fact Sheet, at 
http://www.cde.ca.gov/afterschool/aspfactsheetapr02.doc, September 3, 2002.] In all, 947 
afterschool programs were funded during the ASESP ’s first two years. 

Working with the California Department of Education, the Education Department of the 
University of California at Irvine conducted evaluations of two academic years of the program, 
from 1999 to 2001, releasing results in February 2002. The evaluation relied on data supplied to 
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the state by participating programs, as required by law, and examined student and parent 

satisfaction with their programs, as well as students’ academic outcomes. 

Findings included; 

> SAT-9 scores of participating students increased faster than those of students statewide. In 
reading, 4.2% of afterschool students moved from out of the lowest 25% of their classes. 
“This increase is more than twice the increase found among all students statewide (1.9 
percent). . .” [Evaluation of California ’s After School Learning and Safe Neighborhoods 
Partnerships Program, Department of Education, University of California at Irvine, February 
I, 2002, page 4.] In math, similar findings: 2.5% of afterschool participants moved out of 
the lowest quartile, compared with 1.9% statewide. [Evaluation, page 6.] 

> Significantly, gains were closely related to individual students’ levels of participation in the 
program. “Among students who participated for more than 150 days (approximately 7.5 
months), there was an increase of 4.9% in students above the 25'^ percentile, an increase 
considerably larger than that found statewide. The scores suggest that, particularly for 
students who participate in the ASLSNPP for substantial periods of time, there is a closing of 
the gap in Math achievement between low-income and other students.’’’’ [Evaluation, page 6. 
Emphasis in original.] 

> “The regular school day attendance of students in the ASLSNPP increased between 1999 and 
2000. Among the ASLSNPP participants who were absent 5 percent or more days in 1999, 
the average increase in attendance was 5.6 days. Among those who were absent 10 percent 
or more days in 1999, the average increase in attendance was 1 1 days. Eor those absent 15 
percent or more days in 1999, the average increase in attendance was 17 days.” [Harvard 
Eamily Research Project, Summary of ASLSNPP Evaluation, 

http://www.gse.harvard.edu/~hfrp/proiects/afterschool/mott/aslsnpp.pdf September 3, 

2002.]' 



The After-School Corporation (TASC) - New York 

TASC supports 262 programs across New York State in more than 280 schools, serving 50,000 
students. TASC’s mission is to enhance the quality, availability and sustainability of after- 
school programming in New York State and change public policy so that every child in every 
community across the nation will have access to free, quality after-school programming by 
2010. TASC began in 1998 with a multi-year challenge grant from the Open Society Institute 
(OSI) of up to $125 million, based on the condition that for every dollar OSI contributes, TASC 
is responsible for raising three dollars in other public and private monies. To date, TASC has 
received $99 million of its original grant from OSI and leveraged $300 million in public and 



* The Harvard Family Research Project (HFRP) maintains a rich database of information on out-of-school-time 
evaluations, from which much information in this report is drawn. A complete listing of HFRP’s summaries is 
available at http://www.gse.harvard.edu/hfrp/projects/aflerschool/evaldatabase.html. 
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private funds to support TASC programs. TASC provides grants to nonprofit organizations to 
establish partnerships with individual publie sehools, and the resulting aftersehool programs 
follow a core set of program components. 

The Charles Stewart Mott Foundation and the Carnegie Corporation, joined by the W.T. Grant 
Foundation and the Atlantic Philanthropies provided funding to the Washington-based Policy 
Studies Associates to conduct a five-year evaluation, including annual summary reports. The 
first year’s evaluation, covering the 1998-99 school year, focused largely on issues related to 
program design and participation. The second- and third-year evaluations focused more on 
academic achievement. 

The second-year evaluation, covering the 1999-2000 school year, found; 

> “Students reported feeling safe, relaxed, happy, and connected to their after-school program. 
When asked what they especially liked, they mentioned being with their friends, completing 
their homework before going home, and participating in activities that differed from those of 
the regular school day.” [Building Quality and Supporting Expansion of After-school 
Projects, Summary of Findings, http://www.tascorp.org/pages/promising es2.pdf , page 12.] 

> The evaluation “found significant differences in proficiency-level shifts among active 
participants and nonparticipants who scored in the lowest proficiency level on the 1998-99 
mathematics tests. In math, 3 1 percent of active participants scoring at the lowest proficiency 
level in 1998-99 scored at a higher proficiency level in 1999-2000, compared to 23 percent of 
nonparticipants who demonstrated the same improvement. Two percent of these active 
participants increased their performance to grade level, compared to 1 percent of 
nonparticipants. A similar but less pronounced pattern was observed on the reading tests 
administered in grades 3-8. Among those scoring at the lowest proficiency level in 1998-99, 
45 percent of active participants improved their scores in 1999-2000 enough to move to a 
higher performance level, and 3 percent scored at grade level. Forty percent of 
nonparticipants who scored at the lowest proficiency level in 1998-99 increased their scores 
enough to move to a higher proficiency level a year later, and 2 percent reached grade level.” 
[Patterns of Student-Level Change Linked to TASC Participation Based on TASC Projects in 
Year 2, Executive Summary, 

http://www.policvstudies.com/studies/vouth/Y2%20Performance%20Executive%20Summar 
V-Pdf, page 4.] 

> “Forty- five percent of principals in Year 2 reported that the TASC project has increased 
parents’ attendance at school events and 36 percent said that the project had increased 
parents’ attendance at parent-teacher conferences.” Ninety-seven percent of parents 
surveyed indicated that “their child liked to come to the program”; 86 percent agreed “that 
the project was helping their child academically.” Parents also said that the program helped 
them balance work and family life; 94 percent said the program was convenient; 60 percent 
said they missed less work than before because of the program; 59 percent said it supported 
them in keeping their job; and 54 percent said it was supportive to them in allowing them to 
work more hours. [Building Quality, page 15.] 

The third year of the evaluation, covering the 2000-2001 school year, concluded; 
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> “Students who were aetive participants in TASC projects for more than a year showed 
significantly greater gains on citywide math tests than did similar nonparticipating 
classmates. Students who participated in TASC after-school activities the most consistently 
and for the longest period of time experienced the greatest math gains, when compared to 
similar nonparticipants. Among students who participated actively in TASC projects in each 
year of their enrollment, students participating for two years gained an average of four scale - 
score points more on the city-wide standardized tests than similar nonparticipants. Among 
active participants, students participating for three years gained six points more than similar 
nonparticipants. Demonstrating the value of even higher levels of participation, students 
classified as ‘highly active’ (participating 80 percent or more of the days they were enrolled 
in the year and at least 80 days) gained six scale-score points more than similar 
nonparticipants after only two years of TASC participation. The performance of TASC 
participants on the citywide tests of reading and English/language arts was not significantly 
different from that of similar nonparticipants.” \ What Have We Learned from TASC’s First 
Three Years? Evaluation of the TASC After-School Program, December 2002, page 7, at 
http://www.tascorp.org/pages/psaYear3.pdf 1 

> “In general, the TASC participants who were at greatest academic risk made the largest math 
gains, when compared to other students. (Reporting of subgroup analyses focuses here on 
math because of the consistent relationships with TASC participation, as found in the 
aggregate analyses of math achievement.) Math benefits were clearly evident for students 
who scored in the lowest of four proficiency levels in the year prior to TASC participation. 
The gains for these low-achieving students were evident for active participants regardless of 
their number of years of participation. Among students from low-income families, the 
evaluation also found evidence of after-school benefits in math after two or more years of 
active participation.” [What Have We Learned, page 7.] 

> “Among the various subgroups examined, African-American students were especially likely 
to benefit from active participation in TASC projects, demonstrating gains in math over 
similar nonparticipants after one or more years of active participation. Hispanic students 
benefited in math after two years of participation.” [What Have We Learned, page 7.] 

The final installment of the Policy Studies evaluation of TASC covered data from four full years 

of program operations. When all was said and done, researchers concluded: 

> School principals reported significant benefits for students who participated in TASC 
programs. 95% said that TASC gave students access to activities not available during the 
regular school day, 79% said that participants’ parents expressed more positive feelings 
about the school than before, and 66 percent said that after-school participants received 
special opportunities to hone literacy skills. [This and subsequent quotations from this study 
are from Building Quality, Scale, and Effectiveness in After-School Programs: Major 
Findings of the TASC Evaluation, Policy Studies Associates, Inc., page 3, available at 
http://www.policystudies.com/studies/youth/TASC%20Summary%20Report%20Final.pdf] 

> Parents also expressed satisfaction with TASC services, based, as reported in surveys 
administered to a convenience sample of parents, on their children’s enjoyment of program 
activities, their children’s completion of homework before coming home, and the reassurance 
of knowing that their children were safe and cared for after school, which allowed parents to 
work more hours and miss work less often. 
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> Analyses of data on academic performance and school attendance show that participation in 
TASC activities was linked to improvements in both areas, especially for students who 
participated regularly in TASC programming over two consecutive years. At the elementary- 
and middle-grades level: 

■ Analyses indicate after-school benefits in mathematics achievement, especially for 
those participants who attended TASC programs regularly and for more than a 
year. . . . Comparable findings in reading and English language arts were not apparent, 
although participants in some programs consistently out-gained comparable 
nonparticipants in this area. 

■ Participants showed greater gains in school attendance than did nonparticipants, with 
the greatest benefits at grades 5-8. 

At the high school level: 

■ After-school participants passed more Regents exams and earned more high school 
credits than nonparticipants, but conclusions from this evidence are limited by the 
fact that high school participants were already achieving at substantially higher levels 
than nonparticipants before entering a TASC program. 

■ The difference that reflects the clearest association with TASC high school 
participation is in improved school attendance. When compared to nonparticipants 
with similar records of prior school attendance, TASC participants showed 
significantly more positive school attendance after a year of TASC participation 
(although the attendance of both participants and nonparticipants declined over grades 
9-12). For example, the difference in the attendance gain of participants and similar 
nonparticipants in the lowest attendance quartile during the year before enrolling in 
TASC was +2.4 percentage points over one year, or the equivalent of a net gain of 
+4.4 school days in a 181-day school year (effect size of +0.27). 



North Carolina’s ‘Support Our Students’ 

In 1994, the state of North Carolina launched its “Support Our Students” initiative (SOS), to 
provide funding for afterschool programs across the state. The program offers grants in the 
$60,000 to $250,000 range to nonprofit organizations in the state - one per county, each of 
which coordinates services in their counties. In 2001-2002, the program provided $12.5 million 
to nonprofits in 98 counties. In all, the program supported programs in 190 school-based sites, 
and 54 community-based sites, providing afterschool services to 16,000 students during the 
school year, and summer programming for 10,000 students. 

An evaluation of the 2001-2002 year’s programs, conducted by EDSTAR, an independent 
research and analysis firm based in Raleigh, North Carolina, found the following: 

> “At every grade except sixth grade, improvements in SOS participants’ mean EOG [End of 
Grade Achievement Test] reading scale scores exceeded the state’s improvement goals.” 
[North Carolina Support Our Students 2001-2002 Program Highlights, at 
http://www.edstar.org/sos 2002reports/021009 SOS handout.docl 
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> “Classroom teachers reported that more than 40 percent of the regularly attending 
participants improved their grade in English and/or math.” 

> “The percentages of students who scored at grade level proficiency increased in both reading 
and math, with the greatest increase in reading — from 67 percent at grade level to 7 1 
percent.” 

> “Minority students made greater improvements than White students in both math and 
reading.” 

> “African Americans made greater improvements in reading than any other demographic 
group; minority participants overall made nearly twice the gain of White students in reading 
scores.” 

> “Students continued to make steady, consistent improvement each year they participated in 
SOS. The average yearly improvement was slightly less than half a proficiency level.” 

> “Of the three-year SOS participants, more than two thirds had improved at least two 
proficiency levels in reading and math, compared with the year before joining SOS.” 

> “Except for sixth graders, SOS participants who were the furthest behind and had the most 
risk factors (e.g., free/reduced lunch status, single -parent households, etc.) made the greatest 
gains on EOG. Evaluators surmised that sixth graders often had problems making the 
transition to middle school, and recommended that sixth-grade transition programs be 
implemented.” 



Ohio Urban School Initiative School Age Child Care Project 

The Urban School Initiative School Age Child Care Project (SACC) funds a variety of 
afterschool programs in Ohio urban school districts. The University of Cincinnati College of 
Education’s Evaluation Services Center conducted a thorough review of the program’s 1998- 
1999 school year, measuring both project design and its outcomes. Data collection included 
document reviews, observation of programs, surveys and questionnaires. Among the findings; 

> “Ohio Proficiency Tests scores for both 4th and 6th graders showed that SACC children 
exceeded the state-wide percentages of students meeting proficiency standards. SACC 4th 
grade students’ scores exceeded the statewide percentages of students meeting proficiency 
standards in every subject area tested: writing, reading mathematics, citizenship, and science. 
SACC 6th graders exceeded the statewide percentages of students meeting proficiency 
standards in four of the five areas: writing, reading mathematics, and citizenship.” [Harvard 
Eamily Research Project at 

http://www.gse.harvard.edu/~hfrp/proiects/afterschool/mott/osisaccp.pdf , hereafter HFRP- 
Ohio.] 

> “School absence and tardiness were reduced for participating students. First graders who 
were not in a SACC program during kindergarten reduced the number of school days they 
missed from an average of 8 during their kindergarten year to an average of 3 days during 
their 1998-99 1st grade year. Eighth graders who were not in a SACC program during 7th 
grade reduced the average number of school days missed from 18 to 5.” [HFRP-Ohio.] 
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> “Parents participating in interviews or completing surveys felt the programs had positive 
impacts on their families.” [HFRP-Ohio.] 



Local or Program Level Evaluations 



' The YMCA of Greater New York^s Virtual Y Program 

Since spring 1996, the YMCA of Greater New York has run a Virtual Y initiative - a school- 
based afterschool program serving second- through fourth-graders five afternoons a week. In the 
1997-98 school year, the program expanded from its initial ten pilot sites to operate in as many 
as 100 sites, and no fewer than 66 sites, each year since. More than 50,000 children have 
participated. The program is aimed at lower income public schoolchildren, providing a safe 
environment where children receive reinforcement in reading, math, and healthy lifestyles, as 
well as training intended to emphasize such values as respect, responsibility, honesty, and caring. 

The program has undergone several evaluations by the National Center for Schools and 
Communities (NCSC) at Fordham University. (Funding sources for the study have varied.) 
Several years of research were funded by the Charles Hayden Foundation; several commissioned 
by the YMCA; several funded by the New York State Education Department.) In 2005, NCSC 
published an overview of seven years of research, {The Virtual Y Afterschool Program: A Ray of 
Hope for Urban Public Elementary School Children, available at 

http://www.ncscatfordham.org/binarvdata/files/rayofsunshine final.pdf) . The report found: 

> “Across years, teachers reported between 85 and 91 percent of program participants as 
having behavior problems, usually mild. The greatest problems were consistently in the areas 
of task motivation, frustration tolerance, and learning skills.” 

> “In all years, the data showed statistically significant and moderate to large improvements on 
all seven subscales and on the overall behavior scale.” 

> “NCSC analyses found effects for children at each level of behavioral difficulty from mild to 
severe. The likelihood of obtaining chance differences in performance as large as those found 
in this study is less than one percent.” 

> Virtual Y students outperformed a comparison group in attendance and mathematics, 
although not in reading. [Ray of Hope, pages 8 - 9.] 




Generacion Diez -Adams County, Pennsylvania 



Adams County, Pennsylvania’s Generacion Diez (G-10) afterschool program is aimed 
specifically at the children of migrant workers in the area. Among its goals for children and their 
families is improving the academic achievement of the participating through 6*’’ graders. A 

series of studies conducted by Nathaniel R. Riggs and Mark T. Greenberg, of the Prevention 
Research Center at Pennsylvania State University, summarized by the Harvard Family Research 
Project at http://www.gse.harvard.edu/~hfrp/proiects/afterschooFmott/glO.html, identified a 
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number of positive outcomes, interesting also because of the studies’ findings about which 

students benefited most. The most recent study examined data collected between 2001 and 2003. 

Quoting from HFRP’s summary: 

> “From pretest to posttest, participants’ scores increased significantly in reading, spelling, and 
math (p< .01).” 

> “Among the entire sample, more acculturated children showed significantly greater gains in 
reading achievement over the course of the G-10 program (p < .01), as did children whose 
parents reported less engagement in children’s school activities (p < .05). No significant 
differences in achievement were found by parent-teacher contact or family functioning.” 

> “For children younger than 8 years old, the following groups showed significantly greater 
gains in reading achievement: more acculturated children (p < .05), children in families with 
lower family functioning (p < .05), children whose parents reported fewer parent-teacher 
contacts (p < .01), and children of parents who reported less engagement in children’s school 
activities (p < .01). For children 8 years old and older, none of the assessed variables 
predicted change in children’s reading achievement.” 

> “For the entire sample, more acculturated children, children whose families were functioning 
at a lower level, and children of parents who reported less school engagement demonstrated 
significantly greater gains in spelling achievement (p < .05 for each).” 

> “For the entire sample, children whose parents reported less engagement with their children’s 
school activities at pretest showed significantly greater gains in math achievement across the 
program year (p < .01). No significant results were found for acculturation, family 
functioning, or parent-teacher contacts.” 

> “For children younger than 8 years old, children of parents who were less engaged in school 
activities showed significantly greater gains in math achievement (p < .05). No significant 
results were found for acculturation, family functioning, or parent-teacher contacts.” 

> “For children 8 years and older, children from lower functioning families and children whose 
parents reported less with their children’s school activities showed significantly greater gains 
in math achievement (p <.05 and p <.01, respectively). No significant results were found for 
acculturation or parent-teacher contacts.” 



' Mahoney and Lord Ecological Analysis of After-School Program Participation 

One important and increasingly appreciated challenge researchers confront in evaluating the 
success of afterschool programs is sorting out the relative effects of the many different afternoon 
arrangements that parents have settled on for their children. Some children spend all five 
afternoons a week in an afterschool program, others spend all five in the care of parents or 
another adult. But many children spend their afternoons in some combination of settings, often 
shuttling from an afterschool program to some other care. 

In July 2005 Joseph Mahoney and Heather Lord of Yale University and Erica Carryl of New 
York University released the first of several stages of a study aimed at sorting through and 
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comparing the effects of these different care arrangements, with an eye toward isolating the 
effects of specific arrangements. This first installment of a longitudinal study focused on 599 
“students enrolled in the first, second, and third grades of three public schools in the 
Northeastern United States.” \An Ecological Analysis of After-School Program Participation 
and the Development of Academic Performance and Motivational Attributes for Disadvantaged 
Children, Joseph L. Mahoney, Heather Lord, Erica Carryl, in Child Development, July/August 
2005, Volume 76, Number 4, pp. 811-825, p. 813] When completed, the study will track 
afterschool students for four years, comparing students with four different types of aftersehool 
arrangements - those in afterschool programs, those cared for by parents, those eared for by a 
eombination of siblings or in self eare, and those cared for by a eombination of other adults and 
in sibling or self care. 

Students in the study were enrolled in a large urban sehool district, and most lived in poverty. 
The ongoing study is supported by a grant from the National Institute of Child Health and 
Human Development. 

This first installment found that, “The reading achievement of ehildren in ASP [aftersehool 
program] care was significantly higher than of those in each of the three alternative care 
arrangements. Expectancy of success was also significantly higher for children in ASP care 
eompared with children in other adult/non-adult care.” Refining their analysis, the researchers 
also looked at how ehildren’ s level of engagement in an aftersehool program affected aeademie 
outeomes, and concluded that, “Children in the high-ASP-engagement subgroup had 
significantly higher reading achievement than did those in the three alterative care 
arrangements,” while ehildren with low-engagement achieved at higher levels than only some of 
the other three groups. 




Eos Angeles’s Better Educated Students for Tomorrow, or EA’s BEST, is among the largest and 
best known aftersehool programs in the nation. Eaunched in 1988 as a partnership between the 
Los Angeles Unified School District (LAUSD), the city of Los Angeles and the private sector, 
the program operates at 1 17 sehool sites, serving more than 19,000 students. Schools are ehosen 
for participation because of the generally low academic achievement among their students, or 
because of the low economic status of the community, or high gang or crime rates in the 
neighborhood. 

Since early in the life of the program, the UCLA Center for the Study of Evaluation has 
conducted evaluation studies. The studies have foeused on a variety of topies, using a range of 
measures. The Center released its separate studies in Mareh 1990, March 1991, July 1991, 
December 1993 and spring 1995. In June 2000, the Center released a comprehensive report 
summarizing each of the five previous studies and adding a rich set of findings based on its five- 
year tracking of the academic performance and school attendance of LA’s BEST students who 
were in 2"‘^ through 5*^ grades in the 1993-94 school year. Then in early 2006, the Center 
released a long-term “effeet study,” specifically focused on the long-term impact on students of 
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their participation in LA’s BEST, taking advantage of the massive data set collected over the 
years. 

In particular, the 2006 study (“Keeping Kids in School; An LA’s BEST Example: A Study 
Examining the Long-Term Impact of LA’s BEST on Students’ Dropout Rates,” 
http://www.cse.ucla.edu/products/newsletters/Keeping Kids In Schoo.doc, ) examined the 
dropout rates of students who had participated in LA’s BEST for at least one year, finding 
statistically significant differences between LA’s BEST students and a similar sample of non- 
participating students. Moreover, the study found that the dropout rates of LA’s BEST students 
who participated most frequently and for the longest period of time were most affected; 

[T]he more regularly the students attended the program, the higher the reduction of the 
hazard of dropping out of school. Most interestingly, LA’s BEST participation also 
appeared to have a significant effect in reducing the hazard of dropping out for low- 
income students; that is, the greater the low-income status at the baseline, the longer 
participation in LA’s BEST will keep these students in school. Since LA’s BEST has 
specifically designed the program to serve low-income students, it is encouraging to see 
the program has a significant effect on the targeted population. 

The 2000 study had similarly positive conclusions. “To study LA’s BEST schools,” the authors 
wrote,” we obtained information about students including ethnicity, gender, language proficiency 
status, eligibility for free/reduced lunch (the proxy for low-income level) and disability status. In 
addition, we collected outcome data including achievement test scores (using either the 
Comprehensive Test of Basic Skills [CTBS] or the more recently adopted Stanford-9 
Achievement Test [SAT-9] in reading, mathematics, and language arts. The rate at which 
students were redesignated as fully proficient in English was also collected. We also obtained 
school absence rates, course-taking patterns and rates of student mobility (moving between 
schools or out of the district).” 

In addition, because of the size of the LA’s BEST program and of the LAUSD school system, 
researchers were able to track an extraordinarily large sample of students and a correspondingly 
large “control” group - more than 4,000 LA’s BEST students and more than 15,000 non- 
participating students. The sheer numbers of students tracked make the data produced highly 
reliable. 

The findings, summarized at http://www.lasbest.org/learn/eval.html with a link to the complete 
study, are powerful evidence of the value of afterschool programming. In short, the study found 
that LA’s BEST participants, defined as students who participated regularly and over a period of 
more than one year, when compared to non-participating students, were absent less from school, 
“show positive achievement on standardized tests in mathematics, reading and language arts,” 
and had “higher language redesignation rates to English proficiency.” [Quoting from LA’s 
best’s summary of the findings, at http://www.lasbest. 0 rg/leam/eval.html .l Specific findings: 

> “[0]ur results show that higher levels of participation in LA’s BEST led to better subsequent 
school attendance, which in turn related to higher academic achievement on standardized 
tests of mathematics, reading and language arts.” [A Decade of Results: The Impact of the 
LA ’s BEST After School Enrichment Program on Subsequent Student Achievement and 



www.afterschoolaUiance.org 



19 



Performance, a longitudinal study report and a synthesis of research begun in 1990 by the 
UCLA Center for the Study of Evaluation with support from the Bandai Foundation and the 
City of Los Angeles. June 2000. Denise Huang, Barry Gribbons, Kyung Sung Kim, 
Charlotte Lee, Eva L. Baker. Page 7.] 

> “Language redesignation rates favored LA’s BEST students when compared with non-LA’s 
BEST students for the cohort analyzed (fourth grade, 1994-1995). Significant differences in 
favor of LA’s BEST students were found for subsequent redesignation rates in Grades 6 and 
8. No significant differences were found in comparing performance for Grades 5 and 7.” 
[Decade of Results at page 8.] 

> “Absence follow-up data for the fifth-grade cohort (1994-1995) showed that students who 
participated in LA’s BEST had significantly fewer absences in Grades 6 and 7, although no 
differences were detected in Grades 8 and 9.” [Decade of Results at page 8.] 

> “Although in the initial year LA’s BEST students began with statistically significant 
mathematics achievement scores lower than those of non-participants, in 1997-1998 those 
differences no longer existed.” [Decade of Results at page 9.] 

Researchers conclude: “From our perspective, it looks as if LA’s BEST is a program that, when 
followed as a regular part of students’ broad educational experience, results in statistically 
important differences in student outcomes. The fact that we can detect any change on 
standardized achievement measures in itself is notable, for most educational interventions are 
unable to show impact on measures not tightly tied to the curriculum, or on follow-up 
achievement after a particular program is over. On a practical level, LA’s BEST needs to focus 
its attention on increasing the attendance of enrolled students. It may be that high-level attenders 
do so because they and their parents are more highly motivated, and this interest transfers to 
achievement. But it is equally likely that coming to school and to the LA’s BEST program 
regularly is the reason for good performance and persisting impact subsequent to leaving LA’s 
BEST.” [Decade of Results at pages 9-10.] 



Citizen Schools 

Since 1995, the Boston-based Citizen Schools has provided afterschool and summer programs 
designed to provide children with “authentic, hands-on learning experiences, supportive 
relationships with adults, and positive youth development opportunities.” The program includes 
help with homework, team-building activities, “apprenticeships,” and more. The program is 
based in public schools, and seeks to coordinate its work with school-day teachers. In fall 2002, 
Citizen Schools expanded beyond its home market, to create programs in San Jose, California; 
Houston, Texas; and Worcester and Framingham, Massachusetts. 

Citizen Schools is now sponsoring a five-year evaluation of its work. The most recent phase of 
the research was released in November 2005 [“Putting Students on a Pathway to Academic and 
Social Success: Phase III Findings of the Citizen Schools Evaluation,” Policy Studies 
Associates, Inc., http://www.policvstudies.eom/studies/vouth/CS%20Phase%20III%202005.pdfl 
found: 
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> “Evidence from the first three years of the evaluation shows that Citizen Schools has been 
successful in attracting and retaining educationally at-risk students and in putting these 
students on a path toward academic and social success. . . . [T]he evaluation found that, for 
most short-term indicators of academic success, Citizen Schools had a positive impact on 
students. In particular, based on data from three cohorts of first-year participants. Citizen 
Schools had a positive impact on school attendance and promotion rates, reduction of 
suspension rates, and seventh-grade MCAS scores on the English Language Arts test. Among 
the majority of first-year students who attended regularly for one year (60 percent of program 
days). Citizen Schools also had a positive impact on mathematics and English grades. No 
impact was found on the seventh indicator of success, the sixth-grade MCAS Mathematics 
test.” 

> “Eor students who continued participating in Citizen Schools for a second or third year in the 
seventh or the eighth grades, analysis also revealed evidence of positive impact. However, 
for this group of repeat participants, more indicators show results that are similar for 
participants and matched nonparticipants. Citizen Schools had a positive impact on the 
mathematics grades of returning students in the seventh grade and a positive impact on 
eighth-grade MCAS Mathematics scores, school attendance, and promotion rates for eighth 
graders.” 

> “Emerging evidence at the mid-point of this evaluation shows that former Citizen Schools 
participants are demonstrating gains in high school following their participation and that 
Citizen Schools is succeeding in moving a group of low-income and educationally at-risk 
participants toward a trajectory of successful high school completion and advancement on to 
college. In particular, former Citizen Schools participants enrolled in high-quality high 
schools at far higher rates than the matched comparison group (72 percent to 32 percent) and 
achieved at higher academic levels in ninth grade (half a grade level higher in mathematics 
and English grades). Although academic performance in ninth grade has not yet been adopted 
by Citizen Schools as a formal outcome measure, it may be appropriate to establish it as an 
outcome for future evaluation phases.” 



Pathways to Progress, St Paul, Minnesota Public Schools 

Operating with a three-year 2 E* Century Community Learning Centers grant, in 2000 the St. 
Paul, Minnesota public schools established Pathways to Progress, an eight-site afterschool 
program. Between 2000 and 2003, the sites served more than 3,000 students, one-third of all 
students enrolled at the eight participating schools. 

Researchers Kyla Wahlstrom, Tim Sheldon, and Ashley Murphy of the Center for Applied 
Research and Educational Improvement of the University of Minnesota 
(http ://education.umn. edu/ C AREI/ default.html) conducted an evaluation of the program, 
assessing its success in achieving its principal objectives, among them, increasing student 
academic achievement. Released in March 2004, the evaluation used a matched-pair technique, 
comparing students who had attended the program for 30 or more days each year with matched 
students who had not participated in the program. It found that: 

> “In reading and math, more Pathways participants scored above the national norm on the 
SAT 10 and fewer students scored in the lowest quartile.” 
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> “Data gathered on both two-year and three-year regular partieipants indieate the groups 
experienced continuous progress in their standardized test scores for both reading and 
math. . . . Regular participants frequently began with lower scores in the years prior to 
Pathways involvement and still completed 2003 with higher scores on the standardized tests 
in both subject areas.” 

> “Pathways students experienced dramatically better school attendance, with participants 
attending 18.44 more school days and missing 9.57 fewer school days than their 
nonparticipant counterparts.” 

> “Middle school students in Pathways generally received better marks in English and math 
and more of the grades received by these students were satisfactory ones - a grade of C 
minus or better.” [Final Evaluation Report, 2F* Century Community Learning Centers, 
Pathways to Progress, St. Paul, Minnesota, page 3.] 



After School Education and Safety Program - Santa Ana, California 

With funding from California’s After School Education and Safety Program, Santa Ana, 
California in 1999 opened afterschool sites in four urban middle schools. The sites “serve 
predominantly Latino students with limited English proficiency and from high poverty 
backgrounds. Although each site’s schedule varied, a typical program schedule included a one- 
hour homework period, a one-hour arts or life skills component, and a one-hour sports 
component.” [Harvard Eamily Research Project summary, 

http://www.gse.harvard.edu/hfrp/proiects/afterschool/mott/saaslsnpp.html, December 2003.] 
Jenel Prenovost, Ed.D., of the University of California, Irvine and the University of California, 
Los Angeles led an evaluation that relied on a quasi-experimental design, comparing the one- 
year results of three groups of students - a control group, a high-dosage treatment group 
(students who attended the program for 38 or more days during the school year), and a low- 
dosage group (students who attended for fewer than 38 days). 

The findings indicated that students in the high-dosage group showed better results than low- 
dosage and control group students. Results included the following from HERP at 
h(ttp://www. gse.harvard.edu/hfrp/proiects/afterschooEmott/saaslsnpp.html) : 

> “No statistically significant differences were found in SAT-9 Reading improvement scores or 
NPR [National Percentile Ranking] scores between treatment and comparison groups from 
1999 to 2000. However, high-dosage participants improved somewhat more than matches; 
this was especially true for eighth-graders, females, and students of limited English 
proficiency.” 

> “No statistically significant differences were found in SAT-9 Math improvement scores from 
1999 to 2000 or NPR scores between treatment and comparison groups. However, high- 
dosage participants improved somewhat more than matches and all program participants 
improved more than the general school population. The subgroups of high-dosage sixth 
graders, high-dosage male participants, and high-dosage LEP [Limited English Proficiency] 
participants also improved more than matches. NPR scores favored high-dosage LEP and 
high-dosage LEP participants when compared to the low-dosage participants.” 
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> “At two of the schools, there were significant differences in SAT-9 Math test scores between 
particular subgroups of program participants and comparison group matches. At the first, 
high-dosage males (p<.038) and high-dosage sixth graders (p<.048) improved significantly 
more that the matches. These same two subgroups also had significantly higher NPR scores 
on the SAT-9 Math test than matches. High-dosage participants at the second school had 
significantly (p<.084) higher NPR scores than low-dosage participants.” 

> “There were significantly (p<.005) fewer days of school missed by high-dosage participants 
(5.56 days) as compared to low-dosage participants (7.46 days) and the matches (6.80 days). 
In addition, high-dosage LEP students missed significantly (p<.002) less school than low- 
dosage participants and the matches. Higher-dosage sixth and eighth graders, on the other 
hand, had higher means in days absent than the matches, although this also was not 
statistically significant.” 

> “The program was associated with a nearly significant (p<.082) difference in improvement in 
school attendance. There was a statistically significant (p<.031) finding that high-dosage 
students improved more in their school attendance (1.36 days) than low dosage students (.32 
days). Also, high-dosage LEP students improved their attendance significantly (p<.05) more 
than low-dosage participants (.29 days) and matches (-.18 days).” 



The Foundations Inc. After-School Enrichment Program 

Eor more than a decade, Eoundations, Inc. has operated extended-day enrichment programs and 
provided technical assistance to other afterschool sponsors. During the 2001-2002 school year, 
Drs. Stephen P. Klein and Roger Bolus of Gansk & Associates, of Santa Monica, California, 
administered pre- and post-tests in mathematics and reading to first- through fifth-grade students 
in 19 Eoundations programs in three states. A summary of the report is available on the 
Eoundations website at http://www.foundationsinc.org/ExtendedDavEolder/conclusions.asp . 

The full report, issued in December 2002, concludes: 

> “Eoundations students made substantial improvements in average scores between the fall 
pretest and spring posttest. In fact, their average score gains in mathematics were somewhat 
greater than what would be expected given the results obtained in CTB/McGraw-Hill’s 
national norm sample. The Eoundations students’ gains in reading kept pace with those 
made in this national norm sample.” {Improvements in Math and Reading Scores of Students 
who Did and Did Not Participate in the Foundations After School Enrichment Program 
During the 2001-2002 School Year, Stephen P. Klein, Ph.D. and Roger Bolus, Ph.D., Gansk 
& Associates, December 2, 2002, page 2.] 

> Over the course of the school year, Eoundations’ afterschool students’ test averages moved 
them up national percentile rankings - by an average of 10 percentile rankings in 
mathematics reading, and an average of 2 percentile rankings in reading. {Improvements in 
Math and Reading Scores, page 10.] 

> Eoundations students fared very well by comparison to non-Eoundations students at the 
studied schools. The mathematics “effect size” difference averaged .39 (representing 
39/100*''® of a standard deviation unit), in afterschool students’ favor. In reading, a similar 
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finding; a .41 effect size advantage for afterschool students. {Improvements in Math and 
Reading Scores, page 14.] 

In September 2003, Drs. Klein and Bolus released a follow-up report, assessing of data from the 

2002-2003 school year, again finding statistically significant improvement. 

> “The major finding from this analysis is that there was a statistically significant improvement 
in scores between the pretest and posttest at every grade level and in every subject. 

Moreover, the amount of gain was greater than what would be expected given the results in 
the national norm group. For example, the average FOUNDATIONS’ student was at the 
40th percentile in mathematics on the pretest and at the 46th percentile on the posttest. The 
corresponding values for reading were 45th on the pretest and 48th on the posttest. Language 
arts was 43rd percentile on the pretest and 48th on the posttest. Results were similar within 
grade levels.” {Improvements in Basic Skills Scores of Students Who Did and Did Not 
Participate in the Foundations After School Enrichment Program during the 2002-2003 
School Year, page 7, Stephen P. Klein, Ph.D. and Roger Bolus, Ph.D., Gansk & Associates, 
Santa Monica, California, available from Foundations, Inc., by emailing 
info@foundationsinc . org. ] 

> “We conducted regression analyses to assess the Foundations program’s overall effect on 
improving the students’ test scores. These analyses predicted a student’s posttest score on a 
test on the basis of that student’s pretest score on that same test, grade level, and “group” 

(i.e.. Foundations versus non-Foundations). These analyses found that the students who 
participated in the Foundations program had about a 5-point higher posttest score (which is 
equivalent to about one tenth of a standard deviation unit) than did similarly situated non- 
participants (see Table 6). All the differences in Table 6 were statistically significant (at p < 
.10).” {Improvements, 2002-03, page 11.] 

> “The students who participated in the Foundations program during the 2002-2003 school 
year made substantial gains in math, reading, and language arts scores between the fall 
pretest and spring posttest. These gains generally exceeded the progress of students in 
CTB/McGraw-Hill’s national norm sample. First graders in the Foundations program did 
particularly well, especially in mathematics. Foundations students also had statistically 
significantly greater gain scores between pretest and posttest than did comparable non- 
Foundations students who were tested under the same conditions and at the same time (see 
Table 6). It is not clear why fourth and especially fifth graders who were not in the program 
gained slightly (but not significantly) more between the pretest and posttest than did 
Foundations students. This anomaly may stem from a selection effect that was not accounted 
for by their pretest scores or perhaps to differences in how well the Terra Nova aligns with 
the Foundations’ curriculum across the different grade levels.” {Improvements, 2002-03, 
page 11.] 



YS-CARE After School Program for California Work Opportunity and Responsibility to Kids 

The Los Angeles County Board of Supervisors established the YS-Care After School Program in 
1999. The afterschool program is “designed to offer a safe environment that includes academic 
assistance, homework help, enrichment activities, recreation, and quality childcare provided by 
caring adults in well-supervised school site environments.” {Evaluation of the YS-CARE After 
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School Program For California Work Opportunity and Responsibility to Kids (CalWORKS), 
March 2002, at http://www.gse.uci.edu/asp/aspeval/resourcesAfSCARE13.pdf, page 5, hereafter 
YS-CARE Evaluation.] The program is targeted at K-5 children attending schools in 
neighborhoods with high concentrations of families reeeiving Temporary Assistance to Needy 
Eamilies (TANE). The Department of Education, University of California at Irvine and Research 
Support Services conducted a formal evaluation of the program, and released findings in March 
2002 . 

The evaluation compared the test scores and behavior of participating students with a 
comparable group of non-participating students. The study’s chief conclusions: 

> “YS-CARE participants had larger gains on SAT-9 Reading and SAT-9 Math scores than 
non-participants.” 

> “YS-CARE participants had larger gains on Reading Achievement than matched non- 
participants.” 

> “YS-CARE participants initially in the lowest decile reading group had significantly larger 
reading gains than matched non-participants.” 

> “YS-CARE participants had significantly lower scores on all Work and Study Habits and 
Citizenship measures at baseline. The partieipants narrowed the gap by the time of the end- 
of-year ratings, with almost half of the initial differenees substantially smaller.” \YS-CARE 
Evaluation, pp. 5-6] 



San Diego’s ‘6 to 6’ Extended School Day Program 

San Diego has developed one of the nation’s most ambitious afterschool programs, with the goal 
of making affordable programs before and after school available to every elementary and middle 
school student in the City of San Diego. Two significant evaluations of the program have been 
conducted, one an interim report by WestED, released in April 2001, the other by Hoffman Clark 
and Associates released in July 2001. Using random sampling of sites, document review, 
interviews, focus groups and site observations, WestED found: 

> Parents expressed high levels of satisfaction with the program, “ineluding their pereeptions 
of the quality of academic enrichment, the degree to which children looked forward to the 
program, communication with staff, success at helping children complete homework, and the 
promotion of positive behavior in children.” [Harvard Eamily Research Project website at 
http://www.gse.harvard.edu/~hfrp/proiects/aftersehool/mott/sd66esdp.pdf , hereafter HERP- 
SD.] 

> “Almost two-thirds of responding parents noticed improvements in their children’s academic 
performance.” [HFRP-SD.] 

Relying on random sampling of program partieipants and reviewing a variety of data, WestED’s 
study concluded: 

> Reading scores for “6 to 6” students improved. Eifty-seven percent of students increased 
their reading scores over the eourse of the studied year, and SAT-9 reading seores increased. 
[HFRP-SD.] 
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> Nearly ten pereent of ehildren moved up into the 25* pereentile or higher in 2000 by 
eomparisonto 1999 reading seores. [HFRP-SD.] 

> Forty-four pereent of students inereased their SAT-9 math seores. [HFRP-SD.] 



^ ^ 

The Afterschool Alliance is a nonprofit public awareness and advocacy organization working to ensure 
that all children and youth have access to quality afterschool programs. More information is available at 
www.afterschoolalliance.org . 
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Appendix A 

Afterschool Evaluations at a Glance 



National Studies 


Study Subject 


Evaluator/Author 


Release 

Date 


Key Findings 


Mid-continent Research 
for Education and 
Learning (McREL) 
Meta-Analysis 


Mid-continent Research 
for Education and 
Learning (McREL) 
preformed a meta- 
analysis of 53 separate 
evaluations. 


January 

2004 


The analysis found “statistically significant positive effects of OST [out of 
school time] on both reading and mathematics student achievement.” Lurther, 
“OST strategies that provide one-on-one tutoring for low-achieving or at-risk 
students have strong positive effects on student achievement in reading.” Also, 
“OST strategies need not focus solely on academic activities to have positive 
effects on student achievement.” 


The Boys & Girls Clubs’ 
Project Learn 


Columbia School of 
Social Work 


2000 


A five-site evaluation of Project Learn over 30 months found that engagement 
in reading, use of verbal skills, writing, tutoring, and the study of geography all 
significantly increased as program involvement increased. Program youth 
increased their average grades by 11% from baseline to the 30-month 
measurement while comparison youth increased their average grades by less 
than 1%. 


State-Level Evaluations 


Y oung Scholars Program 
- North Carolina 


Z Smith Reynolds 
Loundation 


2006 


Young Scholars participants made significant gains on the NC state 
standardized tests in both math and reading. Promotion rates for participants 
increased by 83% and regular school day attendance increased. Participant 
grades also improved, with the number of students receiving P’s decreasing by 
50% while the number receiving A’s and B’s increased by 38%. The study is 
based on data from 2000-2005. 


Texas 2L* Century 
Community Learning 
Centers Programs 


Texas Education 
Agency 


2004 


Strong participation in afterschool correlated with better attendance during the 
regular school day. “Approximately half (48%) of youth who participated in 
three quarters or more of the available 21st CCLC activities missed 5 or fewer 
days of school during the fall semester, compared to 1 7% of youth who 
participated in less than one quarter of the available activities.” 


Massachusetts After- 
School Research Study 
(MARS) examined 78 


Intercultural Center for 
Research in Education 
and the National 


2005 


The MARS study identified program characteristics that are most closely 
related to high quality implementation and explored the links between program 
quality and youth outcomes. Researchers found that staff, especially those 
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afterschool centers in the 
state 


Institute on Out-of- 
School Time 




with a strong educational background, and the presence of a highly qualified 
coordinator or director are key to program quality. 


California’s After School 
Education and Safety 
Program (ASLSNPP) 


University of California 
at Irvine working with 
the California 
Department of 
Education 


2002 


A statewide evaluation of California’s After School Education and Safety 
Program demonstrated mathematics gains closely related to students’ level of 
participation in the program: “The scores suggest that, particularly for students 
who participate in the ASESNPP for substantial periods of time, there is a 
closing of the gap in Math achievement between low-income and other 
students.” 


The After-School 
Corporation (TASC) - 
New York 


Policy Studies 
Associates 


Five-year 

data 

released in 
2004. 
(Reports 
issued each 
year of the 
study.) 


Students who participated in TASC afterschool programs attended school more 
regularly and improved their academic performance. Results were especially 
strong for students who participated regularly in the TASC program. At the 
elementary and middle grades level, TASC participants showed gains in math 
achievement and school attendance. At the high school level, afterschool 
participants passed more Regents exams, attended school more regularly and 
earned more high school credits than their non-participating peers. 


Support Our Students - 
North Carolina 


ED STAR 


2002 


Support Our Students participants’ end of grade achievement test scores 
exceeded the state’s improvement goals and the percentages of students who 
scored at grade level proficiency increased in both math and reading. 
Participants who were the furthest behind and considered the most at-risk made 
the greatest gains. 


Ohio Urban School 
Initiative School Age 
Child Care Project 
(SACC) 


University of Cincinnati 
College of Education 
Evaluation Services 
Center 


1999 


Children in the Ohio Urban School Initiative School Age Child Care Project 
(SACC) had higher test scores than non-participating students statewide. 
“SACC 4th grade students’ scores exceeded the statewide percentages of 
students meeting proficiency standards in every subject area tested: writing, 
reading, mathematics, citizenship, and science. SACC 6th graders exceeded the 
statewide percentages of students meeting proficiency standards in four of the 
five areas: writing, reading, mathematics, and citizenship.” 


Local or Program Level Evaluations 


YMCA of Greater New 
York’s Virtual Y 
Program 


National Center for 
Schools and 
Communities at 
Fordham University 


2005 


Virtual Y students outperformed a comparison group in attendance and 
mathematics. Findings are based on seven years of research. 


Generacion Diez 
Adams County, PA 


Prevention Research 
Center at Pennsylvania 


Data 

collected 


Generacion Diez participants’ test scores in reading, spelling and math 
increased significantly. Students demonstrating greatest gains were from 
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State University 


between 
2001 and 
2003 


families that were less in engaged in school and/or functioning at a lower level. 


Mahoney and Lord 
Ecological Analysis of 
After-School Program 
Participation - 
examining students at 
three unnamed public 
schools in the Northeast 


Joseph Mahoney and 
Heather Lord, Y ale 
University and Erica 
Carryl of New York 
University 


July 2005 
(first 

installment 
of ongoing 
evaluation) 


In a study comparing afterschool students with those cared for by parents, by a 
combination of siblings or in self care, and by a combination of other adults 
and in sibling or self care researchers found that children who were in the 
highest category of afterschool program attendance had significantly higher 
reading achievement than children in all other care arrangements. 


LA’s BEST 
Los Angeles, CA 


UCLA Center for the 
Study of Evaluation 


2006 (most 
recent 
release of 
decade-long 
study with 
series of 
reports 
issued) 


The newest report issued from the study found that LA’s BEST participants 
were 20% less likely to drop out of school than non-participants. Previous 
reports in the evaluation show that students’ regular school-day attendance 
improved once they began participating in the afterschool program. That led to 
higher academic achievement on standardized tests of math, reading and 
language arts. In addition, language redesignation rates favored LA’s BEST 
students when compared with non-LA’s BEST students. 


Citizen Schools - sites in 
Massachusetts, Texas 
and California 
Boston, MA 


Policy Studies 
Associates 


November 
2005 (most 
recent 
installment 
of ongoing 
five-year 
evaluation 
report) 


Students participating in Citizen Schools for one year showed positive 
academic outcomes, including school attendance, promotion rates, suspension 
rates, and scores on the Massachusetts Comprehensive Assessment Systems's 
(MCAS) Language Arts test. Second- and third-year students had higher 
mathematics grades, higher scores on the MCAS Mathematics test scores, and 
better school attendance and promotion rates. Students who participated in the 
program went on to achieve higher academic levels in ninth grade and enrolled 
in high-quality high schools at far higher rates than their peers. 


Pathways to Progress - 
St. Paul, MN 


Center for Applied 
Research and 
Educational 
Improvement, 

University of Minnesota 


March 2004 


Pathways participants outperformed comparison students on standardized tests 
in reading and math, received better grades in English and math and attended 
school more regularly. The study is based on data collected between 2000 and 
2003. 


Santa Ana’s After School 
Education and Safety 
Program - 
Santa Ana, CA 


Jenel Prenovost, Ed.D., 
University of 
California, Irvine and 
University of 
California, Los Angeles 


2001 


Students participating 38 days or more in the program improved their SAT-9 
reading and math scores and improved their school attendance, as compared to 
non-participating students or those who attending fewer than 38 days. The 
study analyzed data collected between 1999-2000. 
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Foundations, Inc. - 
Pennsylvania and New 
Jersey 


Gansk & Associates 


2003 (most 
recent 
release in 
series) 


Foundations, Inc. students at every grade level scored higher on math, reading, 
and language arts standardized tests than the comparison group. The study is 
based on data collected from 2001-2003. 


YS-CARE After School 
Program for California 
Work Opportunity and 
Responsibility to Kids 
(CALWORKS) - Los 
Angeles, CA 


Department of 
Education, University 
of California at Irvine 
and Research Support 
Services 


March 2002 


UC-Irvine Department of Education and Research Support Services found that 
reading and mathematics gains of students in Los Angeles’ YS-CARE 
program, aimed at children from families on TANF, outpaced those of non- 
participating students, as measured by SAT-9 scores. 


San Diego’s “6 to 6” 
Extended School Day 
Program - 
San Diego, CA 


Two reports; one by 
WestED and one by 
Hoffman Clark and 
Associates 


2001 


WestED reported that reading scores for San Diego’s “6 to 6” students 
improved. Fifty-seven percent of students increased their reading scores over 
the course of the studied year, and SAT-9 reading scores increased. Nearly ten 
percent of children moved up into the 25* percentile or higher in 2000 by 
comparison to 1999 reading scores. Forty-four percent of students increased 
their SAT-9 math scores. 
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